Papers by Sai Praneeth Karimireddy
Psychological Steering in LLMs: An Evaluation of Effectiveness and Trustworthiness (2026.acl-long)
Copied to clipboard
Amin Banayeeanzade, Ala N. Tak, Fatemeh Bahrani, Anahita Bolourani, Leonardo Blas, Emilio Ferrara, Jonathan Gratch, Sai Praneeth Karimireddy
| Challenge: | Using a model with a high degree of emotion and personality control, large language models can be used to control socially interactive interactions. |
| Approach: | They propose a Psychologically-informed benchmark to evaluate LLM steering effectiveness and trustworthiness across emotion and personality domains. |
| Outcome: | The framework establishes the first holistic evaluation of emotion and personality steering, offering insights into its interpretability and reliability for socially interactive applications. |
TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs (2025.emnlp-demos)
Copied to clipboard
Duygu Nur Yaldiz, Yavuz Faruk Bakman, Sungmin Kang, Alperen Öziş, Hayrettin Eren Yildiz, Mitash Ashish Shah, Zhiqi Huang, Anoop Kumar, Alfy Samuel, Daben Liu, Sai Praneeth Karimireddy, Salman Avestimehr
| Challenge: | Generative Large Language Models (LLMs) produce untruthful outputs, referred to as hallucinations, which are often referred as false positives. |
| Approach: | They propose an open-source Python library with over 30 truthfulness prediction methods. |
| Outcome: | The proposed methods span diverse trade-offs in computational cost, access level, grounding document requirements, and supervision type (self-supervised or supervised). |
Reconsidering LLM Uncertainty Estimation Methods in the Wild (2025.acl-long)
Copied to clipboard
Yavuz Faruk Bakman, Duygu Nur Yaldiz, Sungmin Kang, Tuo Zhang, Baturalp Buyukates, Salman Avestimehr, Sai Praneeth Karimireddy
| Challenge: | Existing studies evaluate UE methods in short-form QA settings, but real-world deployment presents several challenges. |
| Approach: | They examine UE methods' sensitivity to decision threshold selection and their robustness to query transformations such as typos and adversarial prompts. |
| Outcome: | The proposed methods exhibit robustness against typos, adversarial prompts, and prior chat history, and are highly susceptible to adversarials. |
A Systematic Analysis of Base Model Choice for Reward Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) and reward modeling are key to training powerful large language models (LLMs). |
| Approach: | They propose to combine RLHF and reward modeling to boost model selection . they also demonstrate that a small set of benchmarks could be combined to boost the model selection. |
| Outcome: | The results show that the model selection can be improved by up to 14% compared to the most common (default) choice. |